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1  INTRODUCTION 


1  Introduction 

Klondike  Solitaire  is  also  known  as  Patience,  or  simply  Solitaire.  According  to 
Parlett  ([10]  and  [11]),  the  card  game  originated  in  the  eighteenth  century  when 
fortune-tellers  started  cartomancy  (fortune-telling  with  cards).  This  is  supported 
by  the  fact  that  the  earliest  description  of  Patience  occurs  within  a  few  years 
of  the  invention  of  card  layouts  for  cartomancy.  The  popularity  of  the  game  has 
greatly  benefited  from  the  Microsoft  Windows  implementation,  which  has  been 
inseparable  from  every  version  of  Microsoft  Windows  since  1990.  Today,  many 
free  Solitaire  apps  exist  for  playing  on  smartphone  and  tablet.  The  combination 
of  these  factors  makes  it  highly  unlikely,  that  one  is  not  familiar  with  the  game 
of  Klondike  Solitaire. 

Section  1  introduces  the  basic  game  rules  of  Klondike  Solitaire  based  on  [5] 
combined  with  some  well-known  variations  on  these  game  rules.  One  of  these 
variations  is  Thoughtful  Solitaire,  which  probably  has  been  studied  most.  In  this 
variant  the  location  of  each  card  is  known  from  the  beginning  of  the  game:  all 
cards  are  face-up.  Though  a  lot  of  research  on  this  game  variant  has  been  done 
in  [1]  and  [5],  little  information  about  these  results  is  lined  out.  To  give  a  notion 
of  the  complexity  of  the  game,  some  theoretical  results  from  [8]  concerning  the 
complexity  are  presented. 

Section  3  shows  that  despite  the  fact  that  Klondike  Solitaire  is  a  famous  game, 
a  lot  of  uncertainty  still  exists  regarding  some  aspects  of  the  game.  The  exact 
probability  of  winning  a  game  of  Solitaire  and  the  optimal  strategy  are  unknown. 
Research  on  the  amount  of  unplayable  games  has  been  done  in  [6],  [7]  and  [13], 
and  research  on  unsolvable  games  has  been  done  in  [1]  and  [2].  Altogether  some 
research  has  been  done  concerning  all  kinds  of  problems  about  Klondike  Soli¬ 
taire,  but  none  of  them  consider  strategies  that  focus  on  handling  the  cards  in 
the  pile.  In  this  section  we  examine  one  existing  strategy  from  [5]  for  a  game  of 
Kondike  Solitaire.  This  section  also  introduces  two  different,  new  pile-handling 
strategies  for  a  game  of  Klondike  Solitaire. 

To  be  able  to  test  the  effectiveness  of  these  two  different  pile-handling  strategies 
we  created  a  C++  program  which  is  explained  in  Section  4.  This  implementa¬ 
tion  is  used  in  Section  5,  where  some  experiments  concerning  the  pile-handling 
strategies  are  performed. 

This  thesis  is  written  in  pursuit  of  a  Bachelor  of  Science  degree  in  Computer 
Science  and  Mathematics  from  Leiden  University  in  the  Netherlands,  and  has 
been  supervised  by  Walter  Kosters  and  Floske  Spieksnra. 
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2  THE  GAME 


2  The  game 

This  chapter  will  explain  the  game  of  Klondike  Solitaire.  First,  the  basic  game 
rules  are  pointed  out.  Second,  a  few  common  variations  on  these  rules  will  be 
shown.  Third,  we  will  give  a  notion  of  the  complexity  of  the  game. 

2.1  Explanation 

The  traditional  version  of  Klondike  Solitaire  is  played  with  all  52  playing  cards 
in  a  standard  deck  of  cards  without  Jokers.  The  cards  are  divided  into  four 
suits:  two  black  suits  (Spades  4  and  Clubs  4k),  and  two  red  suits  (Hearts  C 
and  Diamonds  ■(>)•  Every  suit  has  13  cards  with  rank  1, . . . ,  13,  where  rank  1 
is  associated  with  Ace  ( A ),  rank  11  with  Jack  (J),  rank  12  with  Queen  ( Q ) 
and  rank  13  with  King  ( K ).  The  game  features  different  locations:  the  pile,  the 
talon,  four  suit  stacks  and  seven  build  stacks.  A  possible  initial  configuration  of 
the  game  is  displayed  in  Figure  1. 


Figure  1:  A  possible  initial  configuration  of  Klondike  Solitaire;  screenshot  from 
[3], 
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2  THE  GAME 


The  game  has  the  following  rules  (formulated  based  on  [5]): 

Build  stacks: 

•  Build  stacks  are  numbered  from  left  to  right,  1, . . . ,  7. 

•  At  the  start  configuration  of  the  game,  the  cards  are  divided  with  one  card 
on  the  first  build  stack,  two  cards  on  the  second  build  stack,  etc. 

•  The  top  card  in  every  build  stack  is  face-up.  The  rest  of  the  cards  in  the 
build  stack  are  face-down. 

•  The  (stack  of)  open  card(s)  on  one  build  stack  is  named  a  card  block. 
A  card  block  can  be  moved  to  another  build  stack,  provided  that  the 
receiving  build  stack  accepts  the  bottommost  face-up  card  of  the  sending 
build  stack.  All  face-up  cards  in  a  card  block  are  to  be  moved  at  once. 
After  moving  a  card  block  to  a  receiving  build  stack,  all  received  cards 
and  already  present  cards  form  a  new  build  stack  together.  The  order  is 
preserved. 

•  If  all  cards  from  a  build  stack  are  moved,  an  empty  stack  remains. 

•  An  empty  stack  can  only  accept  a  King.  The  movement  of  a  King  to  an 
empty  build  stack  is  called  a  Kings  move. 

•  If  the  upper  card  in  a  build  stack  is  face-down,  it  automatically  turns 
face-up. 

•  The  upper  card  of  a  build  stack  can  be  moved  to  the  top  of  a  suit  stack 
provided  that  the  receiving  suit  stack  accepts  this  card. 

•  A  build  stack  will  only  accept  an  incoming  card  block  if  the  upper  card 
of  the  build  stack  is  adjacent  to  and  braided  with  the  bottom  card  of  the 
card  block.  We  define: 

*  A  card  is  adjacent  to  another  card  with  rank  r  if  the  card  has  rank 
r  —  1. 

*  Two  cards  are  braided  with  each  other  if  they  have  a  different  color. 

Suit  stacks: 

•  Every  suit  stack  corresponds  to  one  suit. 

•  At  the  initial  configuration  of  the  game,  all  suit  stacks  are  empty. 

•  Cards  are  being  accepted  in  ascending  order. 

•  A  suit  stack  can  only  accept  cards  with  the  same  suit. 

•  An  empty  suit  stack  can  only  accept  an  Ace. 

•  If  the  suit  stack  is  not  empty,  it  will  only  accept  cards  of  one  rank  higher 
than  the  upper  card  of  the  suit  stack. 
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Pile  and  talon: 

•  The  pile  initially  contains  24  cards,  the  talon  is  initially  empty. 

•  If  the  pile  is  not  empty,  the  upper  three  cards  in  the  pile  can  be  moved  to 
the  talon  one  by  one.  Then  they  can  be  played  in  a  first-in-last-out  order. 

•  If  the  pile  is  empty,  all  cards  from  the  talon  can  be  dealt  back  to  the  pile. 
This  will  preserve  the  ordering  of  the  cards.  This  move  can  be  done  an 
infinite  number  of  times. 

•  A  topcard  of  the  talon  can  be  moved  to  a  suit  stack  or  a  build  stack, 
provided  that  the  stack  accepts  the  card. 

The  goal  of  the  game  is  to  move  all  cards  to  the  suit  stacks.  If  this  succeeds,  the 
game  is  won. 

If  we  generalize  the  game,  we  use  a  deck  of  cards  containing  sn  cards,  divided 
over  s  suits.  If  the  number  of  suits  is  greater  than  four,  and  even,  half  of  the  suits 
is  black  and  half  of  the  suits  is  red.  In  the  most  common  variant  of  Klondike 
Solitaire  the  number  of  suits  is  four:  two  black  suits  (Spades  4*  and  Clubs  ♦), 
and  two  red  suits  (Hearts  C  and  Diamonds  <0>)-  Every  suit  contains  n  cards 
with  ranks  1, 2,  3, . . . ,  n  —  2,  n  —  1,  n.  Rank  1  is  associated  with  generalized  Ace 
(A),  rank  n  —  2  with  generalized  Jack,  (J),  rank  n  —  1  with  generalized  Queen 
(Q),  and  rank  n  with  generalized  King  ( K ).  The  game  consists  of  the  pile  ini¬ 
tially  containing  p  cards,  talon,  s  suit  stacks  and  b  build  stacks.  Depending  on 
variables  s  and  n ,  the  last  build  stack  may  contain  some  remaining  cards  from 
which  the  initial  number  of  cards  in  the  pile  will  be  determined.  An  empty  build 
stack  can  accept  any  card  with  rank  n ,  and  an  empty  suit  stack  can  accept  any 
card  with  rank  1.  The  rules  remain  unchanged. 


2.2  Variations 

Klondike  Solitaire  has  a  lot  of  proporties  we  can  adjust,  so  the  game  will  be 
slightly  different.  Small  adjustments  can  have  a  major  influence  on  the  prob¬ 
ability  of  winning  the  game.  The  following  variations  are  interesting  and  will 
be  described  according  to  the  differences  between  the  variant  and  the  game  as 
formulated  in  Section  2.1. 

2.2.1  Initial  build  stack  variations 

The  initial  build  stack  configuration  as  described  in  the  rules  in  Section  2.1  can 
be  adjusted  in  all  kinds  of  forms,  for  example  a  rectangular  form  with  the  same 
amount  of  cards  on  all  build  stacks.  If  we  consider  Solitaire  without  a  pile,  we 
have  to  determine  how  to  add  the  remaining  cards  to  the  build  stacks.  Of  course 
more  than  seven  build  stacks  is  a  possibility  as  well.  In  this  thesis  the  standard 
initial  build  stack  configuration  is  observed. 
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2.2.2  Moving  a  card  block 

It  is  only  allowed  to  move  a  complete  card  block  to  another  build  stack.  A 
variation  is  to  allow  movement  of  partial  card  blocks.  The  movement  of  partial 
card  blocks  is  allowed  in  [2]  and  [1],  As  in  [5],  the  option  for  moving  partial 
build  stacks  is  disregarded  in  this  thesis. 

2.2.3  Move  card  from  suit  stack  to  build  stack 

In  some  versions  of  Klondike  Solitaire  it  is  allowed  to  move  the  uppermost  card 
of  a  suit  stack,  back  to  a  build  stack,  provided  that  the  build  stack  accepts  this 
card.  This  option  is  allowed  in  [5],  [1]  and  [2].  We  will  not  discuss  the  details  of 
this  variation,  due  to  simplification  reasons,  though  it  does  allows  for  a  greater 
amount  of  possible  moves. 

2.2.4  Move  cards  from  pile  to  talon 

If  the  pile  is  nonempty,  the  upper  card  can  be  played  to  the  talon.  The  upper 
card  in  the  talon  can  be  played.  The  pile  initially  contains  24  cards.  This  means 
that  all  24  of  these  cards  can  be  used  directly.  This  variant  is  called  deal-1.  An 
example  of  all  playable  cards  can  be  found  in  Figure  2. 
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Figure  2:  Playable  cards  in  pile  in  case  of  deal-1.  Created  with  use  of  [3]. 

A  common  variation  is  deal-3,  which  is  used  in  [5],  [1]  and  [2].  In  this  case,  if  the 
talon  is  empty,  the  3  upper  cards  are  played  to  the  talon  in  a  first-in-last-out 
way.  Only  8  out  of  24  cards  in  the  pile  are  now  directly  playable,  and  there  is 
no  guarantee  that  one  will  eventually  be  able  to  play  all  24  cards  in  the  pile. 
This  reduces  the  number  of  possible  moves,  resulting  in  a  harder  game.  Figure  3 
shows  which  cards  are  directly  playable  in  a  deal-3  situation.  In  this  figure  the 
pile  is  dealt  to  the  talon  from  left  to  right:  the  leftmost  card  is  the  first  card 
in  the  talon.  All  directly  playable  cards  are  shifted  up.  If  we  now  play  the  first 
possible  card,  03,  then  the  number  of  playable  cards  directly  increases.  All  cards 
that  are  now  playable  can  be  found  in  Figure  4.  This  figure  follows  directly  from 
Figure  2:  all  directly  playable  cards  are  shifted  up.  This  method  of  determining 
the  playable  cards  is  a  very  greedy  method.  After  playing  a  single  card  from 
the  pile,  the  complete  set  of  playable  pile  cards  may  be  different.  In  general  the 
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following  holds:  after  playing  the  card  with  index  re  {0, . . . ,  23},  we  shift  all 
cards  to  the  left.  This  means  that  all  cards  with  an  initial  index  higher  than  r 
will  have  their  index  decreased  by  1.  Let  us  assume  there  are  p  cards  left  in  the 
pile.  Then  the  cards  with  the  following  index  become  playable,  directly  after 
playing  card  r: 

•  index  r  +  3 m  <  p,  with  m  =  1,2...,  and  m  <  pp. 

•  index  r  —  1  +  3 m  <  p ,  with  m  =  0, 1 . . .,  and  m  <  p~.'s+1 . 

•  index  2  +  3 m  <  r,  with  m.  =  0, 1 . . .,  and  m  <  pp.  These  cards  were 
playable  before  playing  card  r  as  well. 

•  the  last  card,  provided  it  is  not  caught  by  previous  statements. 

In  some  variants  of  the  game  the  number  of  times  the  pile  is  playable  to  the 
talon  is  finite,  but  this  option  will  be  disregarded. 

|6]^3|9f2|A|KJ4|KJ2|7|6|6|7|aj8|334|2|S9|5[5]1D  * 

.".•■pp. . . . 

index  = 

0  1  2  34  5  67  8  9  10  1 1  12  13  14  15  16  17  18  19  20  21  22  23 

Figure  3:  Playable  cards  (the  prevailing  cards)  in  pile  in  case  of  deal-3.  Created 
with  use  of  [3] . 
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Figure  4:  Playable  cards  (the  prevailing  cards)  in  pile  in  case  of  deal-3,  after 
playing  card  ^3.  Created  with  use  of  [3]. 
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2.2.5  Thoughtful  Solitaire 

Thoughtful  Solitaire  has  the  exact  same  rules  as  Klondike  Solitaire.  The  only 
difference  is  that  in  Thoughtful  Solitaire  the  location  of  each  card  is  known  at 
the  beginning  of  the  game;  all  cards  are  face-up.  This  provides  the  option  to 
use  more  information  about  the  location  of  cards  to  determine  which  move  is 
optimal.  As  reported  in  [5]  a  heuristic  player  (following  practically  the  same 
rules  as  our  simplest  heuristic  strategy  as  formulated  in  Section  3.2.1)  for 
Thoughtful  Solitaire  can  supposedly  win  approximately  13.05%  of  the  games, 
and  with  a  rollout  strategy  (a  rollout  strategy  computes  an  action  that  would 
result  from  an  iteration  of  policy  improvement  applied  to  the  heuristic  policy; 
it  may  be  considered  an  alternative  heuristic  that  improves  the  original),  this 
percentage  gains  to  an  alleged  70.20%.  In  [1]  even  a  percentage  of  82%  is  posed. 
This  implies  a  bound  for  the  number  of  winnable  games  of  Klondike  Solitaire. 
The  used  solving  methods  for  Thoughtful  Solitaire  can  be  used  to  solve  Klondike 
Solitaire.  Monte-Carlo  techniques  are  applied  to  the  random  probability  distri¬ 
bution  of  the  closed  cards  in  Klondike  Solitaire.  With  such  a  solver,  using  the 
same  heuristic  as  [5],  [2]  claims  36.97%  of  the  games  of  Klondike  Solitaire  can 
be  won.  Thoughtful  Solitaire  is  not  further  investigated  in  this  thesis. 

2.3  Complexity 

The  problem  of  determining  whether  an  n-card  Klondike  initial  configuration 
can  lead  to  a  win  is  complex.  This  decision  problem  is  referred  to  as  Klondike. 
Problems  discussed  in  Chapter  3  are  a  result  of  this  complexity.  The  great 
complexity  of  this  problem  has  been  extensively  researched  in  [8].  Here  the 
following  definitions  are  used. 

Definition  2.1  (SOLIT(6,  r)).  Let  an  initial  6-black-suit  and  ?’-red-suit 
Klondike  configuration  involving  the  same  number  n  of  cards  per  suit  be  given. 
Determine  whether  the  (6  +  r)n  cards  can  be  placed  on  the  b  +  r  suit  stacks 
by  applying  the  standard  Klondike  game  rules  starting  from  the  given  initial 
configuration. 

Definition  2.2  (FLAT-SOLIT(6,  r)).  SOLIT(6,  r)  with  an  initial  configuration 
having  an  empty  pile  and  empty  talon. 

Definition  2.3  (FLAT-SOLITNoKingC^,  r)).  FLAT-SOLIT(6,  r)  played  with  mod¬ 
ified  rules  that  forbid  an  empty  stack  from  accepting  a  King. 

In  [8]  the  following  complexity  results  about  the  different  types  of  Klondike 
Solitaire  are  proven: 

•  Klondike  is  NP-complete  and  remains  so  with  only  three  suits  available. 

•  Klondike  with  a  black  suit  and  a  red  suit  is  NL-hard. 

•  Klondike  with  any  fixed  number  6  of  black  suits  is  in  NL.  The  same 
holds  for  any  fixed  number  of  r  red  suits. 
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•  Klondike  with  a  single  suit  is  in  AC0  [3] . 

•  Flat  Klondike  (Klondike  Solitaire  with  an  empty  pile)  is  NL-complete 
for  an  arbitrary  number  of  suits. 

•  Flat  Klondike  with  2  black  suits  and  without  Kings  is  in  A C° .  The  same 
holds  for  2  red  suits  without  Kings. 

These  statements  give  a  notion  of  how  complex  the  practical  problems  stated 
in  Chapter  3  are.  Precise  definitions  of  all  used  complexity  classes  can  be  found 
in  [8]  and  [9] .  Cited  from  [8]  we  have: 

AC0  C  AC°[3]  CLCNLCPCNP. 

Flere,  the  complexity  class  AC0  is  the  set  of  languages  accepted  by  DLOGTIME- 
uniform  unbounded- fan- in  constant  depth  {A,  V,  -^-circuits  of  polynomial  size. 
The  larger  class  AC0  [3]  is  the  set  of  languages  AC°-Turing  reducible  to  the 
MODm  Boolean  function,  defined  to  output  1  if  and  only  if  3  does  not  divide 
the  sum  of  its  Boolean  inputs.  The  classes  L  and  NL  stand  for  deterministic 
and  nondeterministic  logarithmic  space  respectively.  The  classes  P  and  NP  are 
deterministic  and  nondeterministic  polynomial  time  respectively. 


3  Game  problems 

Despite  the  fact  that  Klondike  Solitaire  is  a  famous  game,  there  is  still  a  lot 
of  uncertainty  regarding  some  aspects  of  the  game.  The  exact  probability  of 
winning  a  game  of  Solitaire  and  the  optimal  strategy  are  unknown.  Also,  little 
research  has  been  done  to  establish  the  significance  of  the  way  of  handling  the 
cards  in  the  pile.  In  this  section  the  probability  of  winning  a  game  and  different 
strategies  to  win  a  game  are  examined.  The  strategies  focus  mainly  on  finding 
a  strategy  that  handles  moving  of  the  pile  cards.  In  Section  5  some  experiments 
concerning  these  strategies  will  be  performed.  This  entire  section  will  concern 
standard  games  of  Unthoughtful  Klondike  Solitaire,  so  the  variant  of  Thoughtful 
Klondike  Solitaire  explained  in  Section  2.2.5  is  disregarded. 

3.1  Unwinnable  games 

The  initial  configuration  of  Klondike  Solitaire  is  determined  by  distributing  all 
available  cards  randomly  to  all  allowable  initial  locations.  This  has  no  further 
restrictions,  so  it  is  possible  that  a  game  of  Klondike  Solitaire  is  unwinnable. 
We  distinguish  two  types:  unplayable  games  and  unsolvable  games.  Unplayable 
games  of  Klondike  Solitaire  are  games  for  which  the  initial  state  (the  state  of  the 
game  in  the  initial  configuration)  of  the  game  is  unplayable:  there  are  no  possible 
moves.  Unsolvable  games  of  Klondike  Solitaire  are  games  which  can  never  be 
won:  the  game  will  eventually  result  in  an  unplayable  game  state  which  is  not 
necessarily  equal  to  the  initial  game  state.  It  directly  follows  that  unplayable 
games  are  automatically  unsolvable. 


3.1  Unwinnable  games 
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3.1.1  Unplayable  games 

The  initial  situation  of  a  traditional  game  of  Klondike  Solitaire  with  deal-3 
consists  of  15  cards  from  which  one  card  is  potentially  moved  in  the  first  move. 
These  moveable  cards  consist  of  the  7  top  cards  of  each  build  stack  and  the 
8  playable  cards  in  the  talon.  It  follows  that  the  number  of  different  initial 
configurations  equals: 

.  =  28, 837, 689, 349, 669, 200  «  3  •  1016. 

If  one  of  these  15  cards  is  an  Ace,  the  first  possible  move  is  a  fact.  This  Ace  can 
be  moved  to  a  suit  stack  directly.  The  probability  that  none  of  these  15  directly 
playable  cards  contain  an  Ace,  equals: 


However,  this  will  not  cover  all  unplayable  games.  It  may  be  possible  that  2  out 
of  the  7  playable  cards  on  the  build  stacks  can  be  stacked.  This  is  only  the  case  if 
the  cards  are  adjacent  and  braided.  Another  option  is  that  one  of  the  8  playable 
cards  in  the  talon  can  be  moved  to  a  build  stack.  The  following  conditions  have 
to  hold  simultaneously  in  case  of  an  unplayable  game: 

•  There  is  no  Ace  among  the  15  playable  cards. 

•  None  of  the  7  playable  build  stack  cards  can  be  moved  to  another  build 
stack. 

•  None  of  the  8  playable  talon  cards  can  be  moved  to  a  build  stack. 

Latif  [7]  used  Monte  Carlo  simulations  to  show  that  the  percentage  of  unplayable 
games  equals  approximately  0.25%.  Donkersteeg  [6]  shows  that  this  percentage 
can  be  determined  exactly,  by  using  brute  force,  from  which  it  follows  that  the 
number  of  unplayable  games  equals  72,  099,  595, 172, 416.  This  equals  a  percent¬ 
age  of  0.25002%.  De  Ruiter  [13]  confirms  this  percentage  by  dynamic  program¬ 
ming. 

3.1.2  Unsol vable  games 

The  exact  percentage  of  unsolvable  games  of  Klondike  Solitaire  is  still  unknown. 
In  the  literature,  [1]  states  that  no  less  than  82.0%  and  no  more  than  91.4% 
of  Klondike  Solitaire  games  have  winning  solutions,  leaving  the  percentage  of 
unsolvable  games  between  8.6%  and  18.0%.  In  [2]  this  result  is  confirmed.  In 
this  section  we  will  investigate  the  unsolvability  of  games  of  Klondike  Solitaire, 
depending  on  variations  in  number  of  cards  and  number  of  suits. 
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One  suit  without  Kings  move  In  the  case  of  one  suit  and  an  empty  pile, 
what  determines  the  winnability  of  the  game  is  whether  the  cards  in  the  build 
stacks  are  in  order.  In  this  situation,  moves  which  move  a  card  from  a  build 
stack  to  a  suit  stack  are  the  only  moves  allowed.  The  only  possibility  to  win  the 
game  is  to  move  each  card  one  by  one  directly  to  the  suit  stack.  To  be  able  to 
play  all  cards  to  the  suit  stack  by  only  using  this  type  of  move,  all  cards  in  the 
build  stack  need  to  be  in  order.  Consider  the  following  game  variation: 

•  An  empty  pile. 

•  The  Kings  move,  moving  a  King  to  an  empty  build  stack,  is  not  permitted. 

•  There  are  b  build  stacks. 

•  The  first  build  stack  contains  b\  cards,  the  second  build  stack  contains  b-2 
cards,  etc. 

•  Every  top  card  in  a  build  stack  is  face-up. 

The  probability  for  such  a  game  to  be  winnable  equals: 

n  1 

P(cards  in  build  stack  all  in  order)  =  II  . :  (3. 1.2.1) 


3.2  Strategy 

An  optimal  strategy  for  winning  Klondike  Solitaire  is  unknown.  However,  some 
notion  about  what  moves  are  favorable  over  others  is  present.  For  example  mov¬ 
ing  an  Ace  or  2  from  a  build  stack  to  a  suit  stack  is  always  without  risk.  A  little 
more  attention  is  needed  for  further  expanding  the  suit  stacks.  In  general  it 
seems  to  come  in  handy  to  keep  the  ranks  of  the  top  cards  at  all  suit  stacks 
close  to  each  other.  If  there  are  multiple  options  in  moving  a  card  block  to  an¬ 
other  build  stack,  it  seems  favorable  to  move  the  card  block  from  the  build  stack 
which  contains  the  most  remaining  face-down  cards.  It  also  seems  unnecessary 
to  move  the  last  card  in  a  build  stack,  if  you  do  not  have  a  King  available  to 
move  to  the  residual  empty  build  stack. 

In  this  thesis  we  observe  three  strategies  for  Klondike  Solitaire.  The  first  strategy 
is  an  existing  strategy  from  [5],  which  we  will  refer  to  as  simplest  heuristic 
STRATAGY.  For  this  strategy  we  try  to  obtain  the  optimal  parameter  values. 
The  second  and  third  strategies  are  new  strategies.  In  the  first  kind,  which  we 
will  refer  to  as  greedy  pile  strategy,  we  play  by  scoring  moves,  meaning 
that  we  will  score  all  possible  moves  and  make  a  decision  based  on  which  move 
has  the  highest  score.  In  the  second  kind,  which  we  will  refer  to  as  point  pile 
strategy,  we  play  the  game  by  scoring  game  states,  meaning  that  we  will  score 
all  game  states  reachable  by  doing  a  single  move  and  choose  the  move  which 
leads  to  the  game  state  with  the  highest  score.  We  will  score  the  states  based  on 
giving  a  high  score  to  a  more  favorable  state.  For  example,  a  game  state  from 
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3.2  Strategy 


3  GAME  PROBLEMS 


which  a  lot  of  moves  are  possible  could  be  favorable  over  a  game  state  with 
only  one  possible  move  left.  By  running  experiments  using  these  two  kinds  of 
strategies,  we  will  gain  more  insight  about  how  to  choose  which  move  is  best. 
Note  that  all  three  strategies  can  be  used  for  deal-1  and  deal-3,  but  in  this  thesis 
we  will  focus  on  deal-3. 

To  be  able  to  determine  if  one  strategy  is  better  than  an  other,  we  have  to 
somehow  validate  a  strategy.  The  strategies  are  measured  by  evaluating  the  end 
state  of  a  game,  which  is  the  state  in  which  the  game  either  gets  stuck  or  is  won. 
The  properties  of  the  end  state  which  validate  a  strategy  are  shown  in  Figure  5. 
For  example  a  better  strategy  might  be  indicated  by  a  higher  number  of  moves 
before  the  game  is  lost.  These  properties  are  only  used  to  report  about  different 
strategies  in  Section  5. 


The  outcome  which  might  indicate  a 
better  strategy: 


If  the  game  is  lost 

Number  of  moves 

High 

Number  of  cards  in  pile 

Low 

Number  of  closed  cards 

Low 

Number  of  cards  in  suit  stack 

High 

If  the  game  is  won 

Winning  percentage 

High 

Number  of  moves 

Low 

General 

Time  per  game 

Low 

Figure  5:  Strategy  validation  properties. 


3.2.1  Standard  scoring  of  moves 

To  be  able  to  program  a  heuristic  player  we  will  have  to  assign  values  to  possible 
moves,  and  eventually  we  will  even  have  to  assign  priorities  to  moves  with  equal 
values.  To  be  able  to  formulate  such  a  score-based  strategy,  we  have  to  make  a 
lot  of  assumptions  and  choices.  The  assumptions  and  choices  explained  in  this 
section  are  mainly  based  on  intuition  gained  from  a  lot  of  game  playing,  and 
on  the  heuristic  strategy  from  [5].  The  scoring  method  in  this  section  will  be 
referred  to  as  the  simplest  heuristic  strategy. 

Value  of  moves  We  use  parameters  to  assign  a  value  to  each  move  based  ou 
the  heuristic  strategy  from  [5]  described  below.  We  use  parameters  for  each  type 
of  score: 

•  The  initial  score  is  0. 
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•  When  a  card  is  moved  from  a  build  stack  to  a  suit  stack,  SC0RE_0  points 
are  gained. 

•  When  a  card  is  moved  from  a  build  stack  to  another  build  stack,  SC0RE_1 
points  are  gained. 

•  When  a  card  is  moved  from  the  pile  to  a  build  stack,  SC0RE_2  points  are 
gained. 

•  When  a  card  is  moved  from  the  pile  to  a  suit  stack,  SC0RE_3  points  are 
gained. 

A  short  overview  of  all  types  of  moves  with  associated  scoring  parameters  can 
be  found  in  Figure  6.  The  move  that  maximizes  the  score  will  be  executed,  in 
case  of  a  tie  priorities  of  a  move  play  a  role.  The  used  priorities  are  described 
below.  In  [5]  the  parameters  in  Figure  6  have  values  SC0RE_0  =  5,  SC0RE_1  =  0, 
SC0RE_2  =  0  and  SC0RE_3  =  5.  This  choice  is  not  motivated  in  the  article. 


Type 

Description 

Scoring  parameter 

0 

From  build  stack  to  suit  stack 

SC0RE_0 

1 

From  build  stack  to  build  stack 

SCORE. 1 

2 

From  pile  to  build  stack 

SC0RE_2 

3 

From  pile  to  suit  stack 

SC0RE_3 

Figure  6:  Allowable  move  types  with  associated  parameters  in  case  of  the  sim¬ 
plest  HEURISTIC  STRATEGY. 


Priority  of  moves  with  equal  score  The  choice  between  moves  according 
to  this  score  system  is  not  unique  if  the  maximum  score  can  be  achieved  by 
different  moves.  To  be  able  to  make  a  decision  about  what  move  to  choose,  we 
introduce  the  priority  as  follows  (based  on  [5]): 

•  If  the  move  transfers  a  card  block  from  build  stack  to  build  stack,  one  of 
the  following  priorities  holds: 

*  If  the  movement  of  the  card  block  means  a  new  card  can  be  turned 
face-up,  assign  a  priority  of  k  +  1,  in  which  k  equals  the  number  of 
face-down  cards  in  the  build  stack. 

*  If  the  move  empties  a  build  stack,  assign  a  priority  of  1. 

•  If  the  move  transfers  a  card  from  talon  to  build  stack,  the  following  priority 
holds: 

*  If  the  moved  card  equals  K,  assign  a  priority  of  1. 

All  other  moves  have  a  priority  of  0.  In  case  of  multiple  moves  with  a  score 
equal  to  the  maximum  score,  and  a  priority  equal  to  the  maximum  priority,  we 
choose  randomly  among  those  moves. 


12 


3.2  Strategy 


3  GAME  PROBLEMS 


3.2.2  Scoring  pile  moves  greedy 

The  way  of  scoring  and  prioritizing  moves  as  described  in  Section  3.2.1  gives 
little  special  attention  to  pile  moves,  since  we  do  not  have  any  priority  rules  for¬ 
mulated  for  them.  The  way  of  handling  pile  moves  is  actually  a  very  important 
part  of  playing,  and  thus  winning,  a  game  of  Solitaire.  We  introduce  a  greedy 
pile  move  strategy,  based  on  a  Monte  Carlo  Tree  Search  like  approach  for  the 
pile  moves.  Monte  Carlo  Tree  Search  is  a  search  method  combining  the  precision 
of  tree  search  with  the  generality  of  random  sampling,  as  explained  in  [4|,  In 
this  greedy  pile  strategy  we  do  not  use  the  scores  of  integer  type  SC0RE_2  and 
SC0RE_3,  but  we  determine  different  values  for  these  scores  for  each  situation  in 
the  pile.  This  results  in  the  following  approach,  reffered  to  as  the  greedy  pile 

STRATEGY: 

We  use  parameters  for  each  type  of  move: 

•  The  initial  score  is  0. 

•  When  a  card  is  moved  from  a  build  stack  to  a  suit  stack,  SC0RE_0  points 
are  gained. 

•  When  a  card  is  moved  from  a  build  stack  to  another  build  stack,  SC0RE_1 
points  are  gained. 

•  When  a  card  is  moved  from  the  pile  to  a  build  stack,  SC0RE_.x  points  are 
gained. 

•  When  a  card  is  moved  from  the  pile  to  a  suit  stack,  SC0RE_:r  points  are 
gained. 

The  variable  SC0RE_t  with  initial  value  of  0  is  determined  as  follows: 

•  Copy  the  complete  game  state. 

•  Execute  the  considered  pile  move  in  the  copied  game  state.  This  will  result 
in  a  game  state  with  nr_pile  possible  pile  moves  (moves  from  the  pile  to 
another  location  which  are  moves  of  type  2  and  3  as  in  Figure  7). 

•  Now  continue  the  game  in  the  following  way  until  there  are  no  possible 
pile  moves  left: 

*  Take  a  random  pile  move  of  the  nr_pile  possible  pile  moves  and 
execute  this  move  in  the  copied  game  state. 

*  Update  SC0RE_a:  by  adding: 

*  SC0RE_N0T_PILE  multiplied  by  the  number  of  possible  not-pile 
moves  in  this  game  state. 

*  SCORE_PILE  multiplied  by  the  number  of  possible  pile  moves  in 
this  game  state. 
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-k  The  resulting  game  state  has  nr_pile  possible  pile  moves.  If  nr_pile 
is  at  least  one,  then  repeat.  Else,  break  and  return  the  value  of 

SCORE.®. 

A  short  overview  of  all  types  of  moves  with  associated  scoring  parameters  can 
be  found  in  Figure  7.  The  move  which  maximizes  the  score  will  be  executed,  in 
case  of  a  tie  a  random  move  of  the  moves  with  the  maximum  score  is  executed. 


Type 

Description 

Scoring  parameter 

0 

From  build  stack  to  suit  stack 

SC0RE_0 

1 

From  build  stack  to  build  stack 

SCORE. 1 

2,  3 

Pile  move 

Variable  SCORE.®  determined  by: 

•  SCORE.NOT.PILE 

•  SCORE.PILE 

Figure  7:  Allowable  move  types  with  associated  parameters  in  case  of  the 

GREEDY  PILE  STRATEGY. 


3.2.3  Scoring  pile  moves  point-based 

In  addition  to  the  greedy  pile  strategy,  we  introduce  a  point-based  pile 
move  strategy.  This  strategy  is  based  on  specific  game  situation  characteristics 
which  are  observed  when  performing  a  specific  pile  move.  We  do  not  use  the 
scores  of  integer  type  SC0RE_2  and  SC0RE_3,  but  we  determine  different  values 
for  these  scores  for  each  situation  in  the  pile.  This  results  in  the  following  ap¬ 
proach,  referred  to  as  the  point  pile  strategy: 


We  use  parameters  for  each  type  of  move: 

•  The  initial  score  is  0. 

•  When  a  card  is  moved  from  a  build  stack  to  a  suit  stack,  SC0RE_0  points 
are  gained. 

•  When  a  card  is  moved  from  a  build  stack  to  another  build  stack,  SC0RE_1 
points  are  gained. 

•  When  a  card  is  moved  from  a  pile  to  a  build  stack,  SCORE.®  points  are 
gained. 

•  When  a  card  is  moved  from  a  pile  to  a  suit  stack,  SCORE.®  points  are 
gained. 

The  variable  SCORE.®  with  initial  value  of  0  is  determined  as  follows: 

•  Copy  the  complete  game  state. 

•  Execute  the  considered  pile  move  in  the  copied  game  state.  This  will  result 
in  a  new  game  state.  Now  update  SCORE.®  by  adding: 
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*  SCORE_FIRST_PILE  multiplied  by  the  number  of  possible  movable  pile 
cards  with  previous  index  equal  to  0,  3,  6,. . .  as  in  Figure  3. 

*  SC0RE_SEC0ND_PILE  multiplied  by  the  number  of  possible  movable 
pile  cards  with  previous  index  equal  to  1,  4,  7,. . .  as  in  Figure  3. 

*  NEW_BUILD_M0VE  multiplied  by  the  number  of  possible  moves  of  type 
1. 

*  NR_SUIT_M0VES  multiplied  by  the  number  of  possible  moves  of  type 
0  or  3,  which  equals  the  number  of  possible  moves  to  the  suit  stack. 

A  short  overview  of  all  types  of  moves  with  associated  scoring  parameters  can 
be  found  in  Figure  8.  The  move  which  maximizes  the  score  will  be  executed,  in 
case  of  a  tie  a  random  move  of  the  moves  with  the  maximum  score  is  executed. 


Type 

Description 

Scoring  parameter 

0 

1 

2,  3 

From  build  stack  to  suit  stack 
From  build  stack  to  build  stack 
Pile  move 

SC0RE_0 

SCORE. 1 

Variable  SCORE.®  determined  by: 

•  SCORE.FIRST.PILE 

•  SCORE.SECOND.PILE 

•  NEW.BUILD.MOVE 

•  NR.SUIT.MOVES 

Figure  8:  Allowable  move  types  with  associated  parameters  in  case  of  the  point 

PILE  STRATEGY. 


4  Implementation 

To  be  able  to  simulate  a  large  amount  of  Klondike  Solitaire  games,  a  C++ 
program  has  been  written.  In  this  section  an  outline  of  the  implementation 
structure  of  this  program  will  be  presented. 

4.1  Storage  and  initialization  of  the  game  state 

All  cards  are  numbered  1,2, .. .  ,sn.  The  used  and  unused  cards  are  saved  in 
arrays  as  follows: 

•  card_nrs  [2]  [MAX_SUITS  *  MAX_CARDS_PER_SUIT] :  contains  the  rank  and 
suit  of  each  card.  In  the  program  all  cards  are  referred  to  with  a  number. 
When  the  rank  or  suit  of  a  card  is  needed,  it  is  retrieved  from  card_nrs. 

•  used_cards  [MAX_SUITS  *  MAX_CARDS_PER_SUIT]  :  contains  all  card  num¬ 
bers  from  which  the  location  is  known;  all  face-up  cards. 

•  unused_cards  [MAX_SUITS  *  MAX_CARDS_PER_SUIT] :  contains  all  card  num¬ 
bers  from  which  the  location  is  unknown;  all  face-down  cards. 
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The  game  state  is  saved  in  arrays  as  follows: 

•  build_stacks  [MAX_STACKS]  [3  *  MAX_CARDS_PER_ SUIT]  :  contains  all  build 
stacks.  All  numbers  of  face-up  cards  are  stored.  The  first  entry  of  each 
stack  is  reserved  for  the  number  of  face-down  cards  in  this  particular  build 
stack.  The  value  3  *  MAX_CARDS_PER_SUIT  ensures  there  is  enough  space 
reserved  for  the  build  stacks  during  the  game. 

•  suit_stack[MAX_SUITS] :  for  each  suit  it  contains  the  highest  card  rank 
present  in  the  corresponding  suit  stack. 

•  pile  [MAX_PILE] :  contains  all  card  numbers  of  cards  in  the  pile. 

•  playable_pile_cards  [2]  [MAX_PILE] :  contains  all  cards  from  the  pile 
which  can  be  directly  played. 

4.2  Variables  for  different  game  rules 

We  want  to  investigate  the  influence  of  certain  game  rules  on  the  game.  To  be 
able  to  do  so,  we  have  to  create  variables  that  determine  the  rules.  These  are  as 
follows: 

•  bool  M0VE_KING:  is  valued  true  if  we  allow  for  a  King  to  move  to  an  empty 
build  stack. 

•  bool  ASCENDING_BUILD_STACKS:  is  valued  true  if  we  want  to  create  as¬ 
cending  build  stacks,  with  one  card  in  the  first  build  stack,  two  cards  in 
the  second  build  stack,  etc.  If  this  variable  is  false,  then  we  want  to  create 
build  stacks  with  equal  height,  where 

*  int  NR_OF_STACKS:  states  the  amount  of  stacks  in  case  of  stacks  of 
equal  height. 

*  int  NR_OF_CARDS_PER_STACK:  states  the  height  of  stacks  in  case  of 
stacks  of  equal  height. 

•  int  DEAL:  is  valued  3  if  we  want  the  pile  to  handle  a  deal-3  move  as  stated 
in  Section  2.2.4.  All  other  integer  values  are  also  possible,  but  only  the 
value  1,  meaning  deal-1,  describes  a  game  which  is  common  to  play. 

•  int  NR_OF_SUITS:  the  number  of  used  suits. 

•  int  NR_OF_CARDS_PER_SUIT:  the  number  of  cards  per  suit. 

•  int  NR_OF_CARDS_IN_PILE:  the  number  of  cards  present  in  the  pile. 
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4  IMPLEMENTATION 


4.2.1  Implementation  of  the  pile 

As  explained  in  Section  2.2.4,  not  all  of  the  cards  in  the  pile  can  always  be  used 
directly.  In  case  of  deal-1,  all  cards  in  the  pile  are  accessible  directly.  But:  in 
case  of  deal-3  each  card  with  index  mod  3  ==  2  is  available  directly,  meaning  the 
cards  with  index  mod  3  ==  1  and  mod  3  ==  0  are  specifically  not  available  di¬ 
rectly.  The  possibilities  to  access  cards  with  index  mod  3  ==  1  and  mod  3  ==  0 
are  described  in  Section  2.2.4. 

In  our  implementation  of  the  game  we  keep  an  array  playable_pile_cards  []  [] , 
containing  all  directly  accessible  cards.  The  array  is  updated  each  time  a  move 
from  the  pile  is  executed. 

4.3  Function  find_possible_moves 

The  function  f  ind_possible_moves  searches  for  all  possible  moves  in  the  given 
game  situation.  The  possible  moves  are  categorized  in  different  types.  The  cate¬ 
gorization  can  be  found  in  Table  6.  The  function  stores  the  following  information 
per  move: 

•  the  type  of  move,  as  stated  in  Table  6. 

•  the  index  of  the  to  be  moved  card  in  the  location  of  origin. 

•  the  new  index  of  the  to  be  moved  card  in  the  location  of  destination. 

In  case  we  use  play_strategy_heuristic  or  play_strategy_evaluation  it 
also  stores  the  associated  score,  and  in  case  of  play_strategy_heuristic  it  ad¬ 
ditionally  stores  the  associated  priority,  f  ind_possible_moves  determines  how 
many  different  moves  are  possible  and  stores  this  integer  in  nr_of  _possbile_ 
moves  as  well. 

4.4  Function  do_move 

The  function  do_move  carries  out  a  certain  move.  It  updates  the  build  stacks, 
suit  stacks,  pile  and  the  directly  playable  pile  cards  to  the  new  situation.  If 
after  performing  a  move  one  of  the  build  stacks  has  a  face-down  card  on  top, 
the  card  automatically  gets  turned  face-up.  In  this  case  the  arrays  used_cards 
and  unused_cards  will  be  updated  as  well. 

4.5  Implementation  of  different  strategies 

To  be  able  to  test  different  strategies,  there  is  a  first  major  distinction  in  strategy 
for  which  two  different  functions  are  used.  For  the  non-random  strategy,  different 
variables  are  used  to  be  able  to  make  slight  adjustments  to  the  strategy. 
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Function  play  strategy  random  The  function  play_strategy_random  de¬ 
termines,  as  long  as  possible,  a  random  move  and  calls  the  function  do_move  to 
execute  the  determined  random  move.  This  iteration  stops  when  there  are  no 
possible  moves  left.  In  this  case  the  game  is  won,  or  it  is  stuck. 

Function  play  .strategy  .heuristic  The  function  play.strategy .heuristic 

determines,  as  long  as  possible,  the  best  heuristic  move  to  carry  out  for  the  sim¬ 
plest  heuristic  strategy  and  calls  the  function  do.move  to  execute  the 
determined  move.  The  best  heuristic  move  is  the  move  with  maximum  score.  If 
this  results  in  more  than  one  possible  move,  the  move  with  maximum  priority 
is  chosen,  (the  exact  scores  and  priority  rules  which  are  used  can  be  found  in 
Section  3.2.1).  If  this  is  ambiguous  it  chooses  a  random  move  amongst  all  moves 
with  maximum  score  and  maximum  priority.  This  iteration  stops  when  there 
are  no  possible  moves  left.  In  this  case  the  game  is  won,  or  it  is  stuck. 

Variables  for  different  strategies  To  be  able  to  test  certain  slight  strategy 
adjustments  for  game  playing,  variables  for  turning  these  strategies  on  or  off  are 
implemented: 

•  bool  MAX.DIFF.SUIT.STACK:  is  valued  true  if  we  prefer  a  move  to  the 
suit  stack  if  the  maximum  difference  between  the  top  ranks  in  the  dif¬ 
ferent  suits  is  restricted.  Of  course,  it  there  is  no  other  possible  move,  a 
difference  greater  then  the  maximum  difference  will  always  be  allowed.  If 
this  difference  does  not  matter  to  us,  this  variable  is  valued  false. 

•  int  NR.OF.MAX.DIFF.SUIT.STACK:  the  maximum  difference  between  top 
ranks  in  different  suit  stacks.  This  value  is  only  used  if  bool 
MAX.DIFF.SUIT.STACK  is  true. 

•  bool  greedy .pile.strategy  is  valued  true  if  we  want  to  use  the  greedy 
pile  strategy  described  in  Section  3.2.2. 

•  bool  point.pile.strategy  is  valued  true  if  we  want  to  use  the  point 
pile  strategy  described  in  Section  3.2.3. 


5  Experiments 

To  perform  experiments  to  investigate  certain  parts  of  Klondike  Solitaire  we  use 
the  program  explained  in  Section  4.  To  be  able  to  determine  the  importance  of 
a  certain  rule  or  part  of  Klondike  Solitaire,  we  compare  the  results  with  a  game 
of  Klondike  Solitaire  played  by  the  basic  rules  as  formulated  in  Section  2.1.  To 
be  able  to  compare  different  strategies  to  each  other,  all  valuations  of  the  end 
state  of  a  game  as  explained  in  Section  3.2  are  used. 

All  experiments  are  run  on  a  Mac  OS  X  10.9.4  with  a  2,4  GHz  Intel  Core  i5 
processor  and  4  GB  1333  MHz  DDR3  memory. 
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5.1  Assumptions  about  game  rules  and  priorities 

The  game,  and  everything  to  it,  logically  completely  depends  on  the  used  rules 
and  the  associated  priorities.  If  more  moves  are  possible,  the  probability  of  win¬ 
ning  will  improve.  But  it  remains  unclear  what  influence  all  kinds  of  variations 
have  on  the  winning  probability.  To  be  able  to  perform  experiments  to  improve 
the  winning  percentage,  we  make  some  assumptions  about  the  game  rules  we 
use.  In  this  section  we  will  explain  certain  assumptions  we  make,  and  under¬ 
pin  these  assumptions  with  some  experimental  results.  We  will  take  a  look  at 
these  differences  based  on  the  simplest  heuristic  strategy  as  formulated 
in  Section  3.2.1  and  we  use  deal-3  if  not  stated  otherwise.  In  these  first  ex¬ 
periments  our  parameter  values  are  based  on  [5]:  SC0RE_O  =  5,  SC0RE_1  =  0, 
SC0RE_2  =  0  and  SC0RE_3  =  5. 

Kings  move;  allowed  or  not?  The  permission  to  fill  empty  build  stacks 
seems  to  be  a  rather  important  part  of  the  game.  The  simulated  win  percentage 
after  1,000,000  games  with  the  Kings  move  allowed,  equals  25.70%.  The  simu¬ 
lated  win  percentage  after  1,000,000  games  with  the  Kings  move  not  allowed, 
equals  0.30%.  So  it  follows  that  allowing  the  Kings  move  is  very  important  to 
the  winning  rate.  If  we  think  about  it,  this  is  easy  to  understand.  If  a  build 
stack  containing  a  King  also  contains  a  card  from  the  same  suit  and  a  lower 
rank  below  this  specific  King,  the  game  can  never  be  won!  So  in  all  experiments 
from  now  on  we  allow  the  Kings  move. 

Way  of  increasing  suit  stacks  During  the  game  it  is  tempting  to  move 
a  card  directly  to  the  suit  stack,  if  possible.  In  the  end,  all  cards  have  to  be 
moved  to  the  suit  stack,  so  why  not  perform  the  move  as  soon  as  one  has  the 
chance  to  do  so?  However,  if  a  lot  of  cards  with  high  rank  are  already  moved  to 
the  associated  suit  stack,  it  gets  harder  to  stack  cards  in  a  build  stack,  which 
reduces  the  chance  of  turning  face-down  cards.  Simulation  of  1,000,000  games 
gives  the  winning  percentages  if  we  simulate  the  maximum  difference  between 
ranks  in  the  suit  stack  in  Table  9.  We  discriminated  a  deal-1  situation  and  a 
deal-3  situation  and  fixed  the  variable  which  indicates  the  maximum  difference 
between  ranks  in  the  suit  stacks,  NR_0F_MAX_DIFF_SUIT_STACK.  It  appears  that 
in  both  the  deal-1  situation  as  in  the  deal-3  situation,  it  is  optimal  to  give 
priority  to  a  move  to  the  suit  stack  if  its  difference  has  a  maximum  difference 
of  2.  So  in  every  experiment  from  now  on  we  will  give  priority  to  moves  which 
keep  this  maximum  difference  of  2  intact. 
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Maximum  difference  between 
ranks  in  suit  stacks 

Winning  % 

Deal-1 

Deal-3 

1 

37.8% 

9.0% 

2 

40.4% 

9.8% 

3 

37.2% 

7.7% 

4 

36.5% 

7.9% 

5 

30.5% 

7.2% 

6 

28.3% 

6.4% 

7 

28.3% 

6.6% 

8 

28.8% 

6.2% 

9 

28.7% 

6.2% 

10 

26.7% 

5.8% 

11 

27.5% 

6.4% 

12 

27.5% 

5.9% 

Figure  9:  Simulated  percentage  of  winnable  games,  with  a  simulated  maximum 
difference  between  card  ranks  in  the  suit  stacks. 


Number  of  cards  in  the  pile  In  the  original  game,  28  of  the  cards  are 
placed  in  build  stacks,  and  the  remaining  24  cards  are  initially  placed  in  the 
pile.  This  gives  rise  to  the  question  why  the  cards  are  distributed  this  way. 
Simulation  gives  us  Figure  10  and  Table  11.  This  shows  a  certain  periodicity. 
If  we  distribute  52  cards  in  increasing  build  stacks,  this  results  in  build  stacks 
of  size  1,  2,  3,  4,  5,  6,  7,  8,  9,  7.  If  we  observe  Figure  10  from  left  to  right, 
the  first  minimum  is  achieved  when  the  number  of  cards  in  the  pile  is  equal  to 
7.  This  number  corresponds  with  the  last  build  stack  containing  only  7  cards. 
The  second  minimum  (seen  from  left  to  right)  occurs  when  the  number  of  cards 
in  the  pile  is  equal  to  7  +  9  =  16.  This  number  corresponds  to  the  last  build 
stack  containing  7  cards  and  the  seconcl-to-last  build  stack  containing  9  cards. 
This  explanation  holds  for  all  minima  in  the  figure.  Inituitively  this  is  clear.  An 
incomplete  build  stack  placed  in  the  pile  will  result  in  a  higher  probability  of 
winning  because  at  least  one  card  is  left  in  the  build  stack,  so  that  the  top  card 
in  this  build  stack  can  be  used  to  stack  cards  on.  If  we  had  placed  all  these  cards 
in  the  pile,  this  would  not  have  been  possible. 
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Figure  10:  Simulated  percentage  of  winning,  increasing  number  of  cards  in  pile. 
Based  on  simulation  of  1,000,000  games. 


Variant 

Winning  percentage 

deal-1 

35.6% 

deal- 2 

21.4% 

deal- 3 

10.2% 

deal-4 

4.1% 

Figure  11:  Simulated  winning  percentage  with  24  cards  in  the  pile,  based  on 
1,000,000  games. 
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5.2  Variations  in  strategy 

In  Section  3.2,  three  types  of  strategies  have  been  discussed:  the  simplest 
HEURISTIC  STRATEGY,  the  GREEDY  PILE  STRATEGY  and  the  POINT  PILE  STRAT¬ 
EGY.  Of  course  there  always  is  the  random  strategy,  in  which  a  random  move  is 
carried  out.  In  this  subsection  we  will  investigate  the  different  types  of  strate¬ 
gies  and  their  influence  on  winning  rates.  Note  that  due  to  reasons  explained  in 
Section  5.1  on  page  19  we  will  allow  the  Kings  move  and  give  priority  to  moves 
which  keep  the  maximum  difference  of  2  between  ranks  in  suit  stacks  intact  in 
all  experiments. 

5.2.1  Random  player 

If  we  simulate  1,000,000  games  played  by  the  random  player  using  deal-1,  the 
winning  percentage  is  about  10.41%.  The  total  experiment  had  a  duration  of 
7.38  minutes.  In  case  of  deal-3  the  winning  percentage  is  about  3.78%,  and 
the  duration  of  the  experiment  was  5.00  minutes.  From  [2]  anecdotal  evidence 
suggests  that  typical  human  players  win  around  15%  of  the  games,  so  we  know 
that  a  good  strategy  will  definitely  increase  the  winning  percentage. 

5.2.2  Heuristic  player 

When  a  game  is  played,  choices  are  made  based  on  some  sort  of  strategy.  A 
heuristic  player  also  uses  a  strategy,  as  explained  in  Section  3.2  and  Section  4.5. 
In  this  section  we  show  the  results  of  all  executed  strategy  experiments. 

Experiments  for  the  simplest  heuristic  player  In  this  paragraph  we  want 
to  find  the  best  values  for  the  parameters  SC0RE_0,  SC0RE_1,  SC0RE_2  and 
SC0RE_3  which  are  used  for  the  simplest  heuristic  strategy  as  formulated  in 
Section  3.2.1.  In  Section  3.2.1  we  have  used  values  5,  0,  0  and  5,  but  because  the 
strategy  is  determined  by  the  ratio  of  the  parameters,  multiplying  them  by  the 
same  integer  results  in  the  same  strategy.  For  this  reason  we  use  the  parameter 
values  multiplied  by  100.  To  search  for  a  good  combination  of  these  scores,  we 
fix  SC0RE_O  at  500  and  run  experiments  for  all  possible  combinations  with  all 
other  score  values  equal  to  one  of  the  values  100,  300,  500,  700,  900  or  1100. 
We  have  tested  216  different  score  combinations  used  to  play  100,000  games  per 
score  combination.  This  experiment  took  about  2.84  hours,  so  using  this  strategy 
takes  about  0.00047  seconds  per  game  of  Solitaire.  These  experiments  led  to  a 
first  result  of  the  best  combination  of  parameters  to  lead  to  a  high  winning 
percentage.  The  combinations  resulting  in  the  top  ten  winning  percentages  are 
picked  and  run  again  for  1,000,000  games  per  combination  for  a  better  validation 
of  the  result.  Results  are  shown  in  Figure  12. 

As  displayed  in  Figure  12,  we  observe  the  10  parameter  combinations  resulting 
in  the  highest  winning  percentages.  The  games  played  with  these  combinations 
give  a  notion  of  the  value  of  the  strategy  as  explained  in  Section  3.2.  The  average 
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SCORE, 1 

SC0RE_2 

SC0RE_3 

winning  percentage 
based  on 
100,000  games 

winning  percentage 
based  on 
1,000,000  games 

100 

100 

100 

9.84% 

9.84% 

100 

100 

300 

9.90% 

9.80% 

100 

100 

500 

9.84% 

9.80% 

300 

300 

100 

9.85% 

9.83% 

300 

300 

300 

9.84% 

9.88% 

700 

100 

100 

9.84% 

9.75% 

700 

100 

300 

9.93% 

9.73% 

700 

300 

100 

9.86% 

9.72% 

900 

100 

500 

9.84% 

9.73% 

1100 

100 

300 

9.90% 

9.76% 

Figure  12:  Parameter  combinations  with  SC0RE_0  =  500  resulting  in  top  10 
highest  winning  percentages  after  playing  100,000  games  and  associated  win¬ 
ning  percentages  for  playing  1,000,000  games  with  the  simplest  heuristic 

STRATEGY. 


values  of  these  results  are  displayed  in  Figure  13.  The  average  number  of  cards 
on  the  suit  stack  in  the  end  state  of  the  game  is  represented  in  Figure  14. 


If  the  game  is  lost 

Number  of  moves 

25.11 

Number  of  cards  in  pile 

12.40 

Number  of  closed  cards 

12.11 

Number  of  cards  in  suit  stack 

2.15 

If  the  game  is  won 

Winning  percentage 

9.78% 

Number  of  moves 

95.30 

General 

Time  per  game  0.00047  s 


Figure  13:  Strategy  validation  properties  of  the  10  best  parameter  combinations 
determining  the  simplest  heuristic  strategy. 

Based  on  these  experiments,  the  parameter  combination  of  SC0RE_0  =  500, 
SC0RE_1  =  300,  SC0RE_2  =  300  and  SC0RE_3  =  300  has  the  best  winning  per¬ 
centage.  This  is  remarkable  because  this  result  implies  that  only  a  move  from  a 
build  stack  to  a  suit  stack  is  more  favorable  over  all  other  kind  of  moves. 


Experiments  using  a  greedy  pile  strategy  In  this  paragraph  we  want  to 
find  the  best  values  for  the  parameters  SC0RE_0,  SC0RE_1,  SC0RE_N0T_PILE  and 
SC0RE_PILE,  which  are  used  for  the  greedy  pile  strategy,  as  formulated  in 
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Number  of  cards  in  suit  stack 

Figure  14:  Average  number  of  cards  in  suit  stack  at  the  end  state.  The  simplest 
heuristic  strategy  is  used  with  parameter  values  equal  to  the  top  10  winning 
combinations. 


Section  3.2.2.  To  search  for  a  good  combination  of  these  scores,  we  fix  SC0RE_0  at 
500  and  vary  all  other  score  values.  We  have  tested  216  different  combinations  as 
shown  in  Figure  15.  These  are  used  to  play  10,000  games  per  score  combination. 
This  experiment  took  about  9.09  hours,  so  using  this  strategy  takes  0.01516 
seconds  per  game. 


SC0RE 1 

SC0RE N0T PILE 

SC0RE PILE 

100 

1 

1 

300 

2 

2 

500 

3 

3 

700 

4 

4 

900 

5 

5 

1100 

6 

6 

Figure  15:  Used  score  combinations  of  parameters  with  SC0RE_0  =  500  for  the 
greedy  pile  strategy. 

These  experiments  led  to  a  first  result  of  the  best  combination  of  parameters 
to  lead  to  a  high  winning  percentage.  The  combinations  resulting  in  the  top 
ten  winning  percentages  are  picked  and  run  again  for  1,000,000  games  per  com¬ 
bination  for  a  better  validation  of  the  result.  Results  are  shown  in  Figure  16. 
As  displayed  in  Figure  16,  we  observe  the  10  parameter  combinations  resulting 
in  the  highest  winning  percentages.  The  games  played  with  these  combinations 
give  a  notion  of  the  value  of  the  strategy  as  explained  in  Section  3.2.  The  average 
values  of  these  results  are  displayed  in  Figure  17.  The  average  number  of  cards 
on  the  suit  stack  in  the  end  state  of  the  game  is  represented  in  Figure  18. 
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SCORE, 1 

SCORE, 

N0T_PILE 

SCORE, 

PILE 

winning  percentage 
based  on 

10,000  games 

winning  percentage 
based  on 
1,000,000  games 

300 

3 

1 

17.58% 

17.76% 

500 

5 

4 

17.65% 

17.76% 

1100 

4 

1 

17.62% 

17.73% 

500 

6 

2 

18.01% 

17.68% 

700 

4 

1 

18.01% 

17.66% 

900 

6 

3 

17.84% 

17.65% 

900 

4 

2 

17.92% 

17.62% 

900 

2 

1 

17.28% 

17.62% 

500 

5 

3 

18.04% 

17.59% 

500 

3 

2 

17.88% 

17.57% 

Figure  16:  Parameter  combinations  with  SC0RE_0  =  500  resulting  in  top  10 
highest  winning  percentages  after  playing  10,000  games  and  associated  winning 
percentages  for  playing  1,000,000  games  with  the  greedy  pile  strategy. 


If  the  game  is  lost 

Number  of  moves 

30.02 

Number  of  cards  in  pile 

10.05 

Number  of  closed  cards 

10.68 

Number  of  cards  in  suit  stack 

3.80 

If  the  game  is  won 

Winning  percentage 

17.76% 

Number  of  moves 

93.86 

General 

Time  per  game 

0.015s 

Figure  17:  Strategy  validation  properties  of  the  10  best  parameter  combinations 
determining  the  greedy  pile  strategy. 
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Number  of  cards  in  suit  stack 


Figure  18:  Average  number  of  cards  in  suit  stack  at  the  end  state.  The  greedy 
pile  strategy  is  used  with  parameter  values  equal  to  the  top  10  winning 
combinations. 


Based  on  these  experiments,  the  parameter  combination  of  SC0RE_0  =  500, 
SC0RE_1  =  300,  SC0RE_N0T_PILE  =  3  and  SC0RE_PILE  =  1  has  the  best  win¬ 
ning  percentage. 

Experiments  using  a  point  based  pile  strategy  In  this  paragraph  we 
want  to  find  the  best  values  for  the  parameters  SC0RE_0,  SC0RE_1,  SC0RE_FIRST 
_PILE.  SC0RE_SEC0ND_PILE,  NEW_BUILD_M0VE  and  NR.SUIT.M0VES,  which  are 
used  for  the  point  pile  strategy  as  formulated  in  Section  3.2.3.  To  search 
for  a  good  combination  of  these  scores,  we  fix  SC0RE_0  at  50  and  vary  all  other 
score  values.  We  have  tested  3125  different  combinations  as  shown  in  Figure  19. 
These  are  used  to  play  1,500  games  per  score  combination.  This  experiment 
took  about  7.46  hours,  so  using  this  strategy  takes  0.0057  seconds  per  game. 
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30 

30 
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40 
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50 

Figure  19:  Used  score  combinations  of  parameters  with  SC0RE_0  =  50  for  the 

POINT  PILE  STRATEGY. 

These  experiments  led  to  a  first  result  of  the  best  combination  of  parameters  to 
lead  to  the  highest  winning  percentage.  The  combinations  resulting  in  the  top 
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ten  winning  percentages  are  shown  in  Figure  20. 
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Figure  20:  Parameter  combinations  with  SC0RE_0  =  50  resulting  in  top  10  high¬ 
est  winning  percentages  after  playing  1,500  games  with  the  point  pile  strat¬ 
egy. 

Some  notable  things  in  these  results  are  the  following: 

•  in  0.02%  of  the  cases,  the  value  of  SC0RE_1  is  greater  than  or  equal  to  100. 

•  the  values  of  all  other  variables  are  almost  always  around  the  value  of  10 
for  the  top  winning  percentages. 

From  this  observations  we  choose  to  do  another  experiment  with  a  fixed  value 
of  SC0RE_0  =  50,  SC0RE_1  =125,  and  varying  the  other  score  parameters  with 
slighter  changes  around  the  value  of  10,  resulting  in  the  top  10  highest  winning 
percentages  displayed  in  Figure  21.  As  displayed  in  Figure  21,  we  observe  the 
10  parameter  combinations  resulting  in  the  highest  winning  percentages.  The 
games  played  with  these  combinations  give  a  notion  of  the  value  of  the  strategy 
as  explained  in  Section  3.2.  The  average  values  of  these  results  are  displayed  in 
Figure  22.  The  average  number  of  cards  on  the  suit  stack  in  the  end  state  of  the 
game  is  represented  in  Figure  23. 

Based  on  these  experiments,  the  parameter  combination  of  SC0RE_0  =  50, 
SCORE.l  =  125,  SC0RE_FIRST_PILE  =  6,  SC0RE_SEC0ND_PILE  =  6, 
NEW_BUILD_M0VE  =  8  and  NR_SUIT_M0VE  =  8  have  the  best  winning  percent¬ 
age. 
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Figure  21:  Parameter  combinations  with  SC0RE_0  =  50  and  SC0RE_1  =  125  re¬ 
sulting  in  top  10  highest  winning  percentages  after  playing  1,500  games  and 
associated  winning  percentages  for  playing  1,000,000  games  with  the  point 

PILE  STRATEGY. 


If  the  game  is  lost 

Number  of  moves 

28.52 

Number  of  cards  in  pile 

10.99 

Number  of  closed  cards 

11.99 

Number  of  cards  in  suit  stack 

3.21 

If  the  game  is  won 

Winning  percentage 

13.75% 

Number  of  moves 

86.64 

General 

Time  per  game 

0.0057  s 

Figure  22:  Strategy  validation  properties  of  the  10  best  parameter  combinations 
determining  the  point  pile  strategy. 


28 


5.3  Summary  of  results 


5  EXPERIMENTS 


18,00% 

16,00% 

^  14,UU% 

«/> 

g  q.  12,00% 

2n  <C 

h—  F  10,00% 

o  re 

|  o  8,00% 

c  ° 

E  o 

o'  6,00% 

Z  o 
© 

ll  I 

2,00% 

0 

8  12  16  20  24  28  32  36  40  44  48  52 

Number  of  cards  in  suit  stack 

Figure  23:  Average  number  of  cards  in  suit  stack  at  the  end  state.  The  point 
pile  strategy  is  used  with  parameter  values  equal  to  the  top  10  winning 
combinations. 


5.3  Summary  of  results 

A  summary  of  the  results  of  playing  a  game  of  Klondike  Solitaire  with  one  of  the 
three  discussed  strategies  is  shown  in  Figure  24.  In  this  figure  a  few  statistics 
are  striking.  In  this  section  the  differences  between  strategies  are  discussed. 


Simplest  h. 

Greedy  pile 

Point  pile 

If  the  game  is  lost 

Number  of  moves 
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30.02 

28.52 

Number  of  cards  in  pile 
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Number  of  closed  cards 
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If  the  game  is  won 
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17.76% 

13.75% 

Number  of  moves 

95.30 

93.86 

86.64 

General 

Time  per  game 

0.00047  s 

0.015s 

0.0057  s 

Figure  24:  Properties  of  every  tested  strategy. 

The  greedy  pile  strategy  has  the  highest  winning  percentage.  If  you  change 
your  strategy  from  simplest  heuristic  to  greedy  pile,  the  winning  rate  im¬ 
proves  81.59%,  but  the  time  needed  to  accomplish  a  win  increases  with  a  factor 
32.  Changing  strategy  from  simplest  heuristic  to  point  pile,  leads  to  a  win¬ 
ning  rate  improvement  of  40.59%  by  having  the  time  increase  with  a  factor  12. 
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A  striking  difference  is  that  the  point  pile  strategy  wins  a  game  using  less 
card  moves.  This  is  explained  by  the  fact  that  the  greedy  pile  strategy 
prefers  game  states  in  which  more  cards  are  present  in  build  stacks  over  more 
cards  present  in  the  pile.  Meaning  that  the  greedy  pile  strategy  tends  to 
move  cards  more  often  from  the  pile  to  a  build  stack,  compared  to  the  point 
PILE  STRATEGY. 

The  experiments  bring  forward  a  very  remarkable  aspect  of  a  lost  game  of 
Klondike  Solitaire.  As  we  can  see  in  Figure  24,  a  game  of  Solitaire  gets  stuck  in 
a  situation  where  the  average  amount  of  closed  cards  is  very  close  to  the  average 
amount  of  cards  in  the  pile. 

5.3.1  Comparison  of  thesis  results  to  literature  results 

While  all  experiments  in  this  thesis  are  based  on  unThoughtful  Solitaire,  all 
results  in  the  literature,  except  from  results  in  [2],  are  based  on  Thoughtful 
Solitaire.  The  difference  between  these  to  variants  is  described  in  Section  2.2.5. 

The  heuristic  player  (a  player  following  a  heuristic)  for  Thoughtful  Solitaire, 
described  in  [5],  can  supposedly  win  approximately  13.05%  of  the  games  with 
an  average  time  of  0.021  seconds  per  game.  With  a  rollout  strategy  (a  roll¬ 
out  strategy  computes  an  action  that  would  result  from  an  iteration  of  policy 
improvement,  applied  to  the  heuristic  policy;  it  may  be  considered  an  alterna¬ 
tive  heuristic  that  improves  the  original),  this  percentage  gains  to  an  alleged 
70.20%.  This  strategy  takes  an  average  1  hour  and  45  minutes  per  game.  All 
strategies  discussed  in  this  thesis  take  fewer  time  per  game  than  the  heuristic 
player  from  [5]  and  all  strategies,  except  the  simplest  heuristic  strategy, 
have  a  higher  winning  percentage  than  the  the  heuristic  player  from  [5]. 

The  heuristic  player  from  [1]  achieves  a  winning  percentage  of  16.17%  in  an 
average  time  of  0.02  seconds  per  game.  This  result  is  quite  close  to  the  results 
of  our  greedy  pile  strategy. 

The  solving  methods  used  for  Thoughtful  Solitaire  can  be  used  to  solve  Klondike 
Solitaire.  Monte-Carlo  techniques  are  applied  to  the  random  probability  distri¬ 
bution  of  the  closed  cards  in  Klondike  Solitaire.  Such  a  solver  is  used  in  [2], 
where  a  claimed  percentage  of  36.97%  of  the  games  of  Klondike  Solitaire  can  be 
won.  This  strategy  takes  on  average  2280.55  seconds  per  game,  which  is  much 
higher  than  the  duration  of  our  strategies.  The  quickest  strategy  in  [2]  takes  on 
average  44.50  seconds  per  game  and  has  a  winning  percentage  of  24.64%  which 
is  a  winning  percentage  none  of  our  three  strategies  comes  close  to. 
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6  Conclusions  and  Further  Research 

We  have  analyzed  three  different  strategies  for  playing  a  game  of  Klondike  Soli¬ 
taire,  two  of  which  have  a  focus  on  pile  moves.  These  pile  moves  appear  to  be 
important  since  games  get  stuck  with  the  average  amount  of  closed  cards  and 
the  average  amount  of  cards  in  the  pile  very  close  to  each  other. 

The  greedy  pile  strategy  has  a  partial  Monte-Carlo  like  greedy  approach  for 
considering  pile  moves,  aside  from  a  simple  heuristic  approach  for  other  kind  of 
moves.  Having  a  different  kind  of  approach,  for  different  types  of  moves  appear 
to  be  useful  since  this  strategy  leads  to  a  winning  percentage  of  17.76%  in  only 
0.015  seconds.  Such  a  high  winning  percentage  in  so  little  time  is  remarkable. 

A  suggestion  for  future  work  is  that  it  could  be  interesting  to  further  split  these 
pile  moves  into  different  types  of  moves  and  search  for  a  different  strategy  for 
each  type  of  move.  As  we  have  seen  in  this  thesis,  pile  moves  are  of  very  high 
importance  in  a  game  of  Klondike  Solitaire,  but  this  greedy  pile  strategy 
is  only  a  start. 
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